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ABSTRACT 

Objective: Examine clinical reasoning and decision 
making in an out of hours (00H) primary care setting 
to gain insights into how general practitioners (GPs) 
make clinical decisions and manage risk in this 
environment. 

Design: Semi-structured interviews using open-ended 
questions. 

Setting: A 2-month qualitative interview study 
conducted in Oxfordshire, UK. 
Participants: 21 GPs working in 00H primary care. 
Results: The most powerful themes to emerge related 
to dealing with urgent potentially high-risk cases, 
keeping patients safe and responding to their needs, 
while trying to keep patients out of hospital and the 
concept of 'fire fighting'. There were a number of 
well-defined characteristics that GPs reported making 
presentations easy or difficult to deal with. Severely ill 
patients were straightforward, while the older people, 
with complex multisystem diseases, were often 
difficult. GPs stopped collecting clinical information 
and came to clinical decisions when high-risk disease 
and severe illness requiring hospital attention has been 
excluded; they had responded directly to the patient's 
needs and there was a reliable safety net in place. 
Learning points that GPs identified as important for 
trainees in the 00H setting included the importance of 
developing rapport in spite of time pressures, learning 
to deal with uncertainty and learning about common 
presentations with a focus on critical cues to exclude 
severe illness. 

Conclusions: The findings support suggestions that 
improvements in primary care 00H could be achieved 
by including automated and regular timely feedback 
system for GPs and individual peer and expert clinician 
support for GPs with regular meetings to discuss 
recent cases. In addition, trainee support and 
mentoring to focus on clinical skills, knowledge and 
risk management issues specific to 00H is currently 
required. Investigating the stopping rules used for 
diagnostic closure may provide new insights into the 
root causes of clinical error in such a high-risk setting. 



INTRODUCTION 

Primary care in the UK is provided during 
weekends, evenings and overnight by various 
types of out of hours (OOH) services, and it 
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Article focus 

■ Clinical reasoning and decision making in an out 
of hours (OOH) primary care setting. 

■ The aim is to gain insights into how general 
practitioners (GPs) make clinical decisions and 
manage risk in this environment. 

■ Implications for system changes and training. 

Key messages 

■ Clinical decision making in OOH is dominated 
by rule-out strategies for severe illness or 
potentially high-risk diseases. 

■ GPs use three main criteria to determine 
diagnostic closure: global wellness with rule- 
outs, responded to patient needs, presence of 
a reliable safety net. 

■ Improvements to clinical decision making could 
be achieved by providing routine feedback to 
clinical staff working in OOH, building in systems 
to support reflection on clinical cases and more 
tailored GP training. 

Strengths and limitations of this study 

■ The design of the study is based on a strong 
theoretical framework provided by the dual 
theory of cognition. 

■ Face validity through using recently seen cases. 

■ Limitations relate primarily to sampling, partic- 
ipants consisting of self-selected individuals. 

has become a core component of round- 
the-clock primary care in many countries. 
Accessible and high-quality OOH services 
provide an important mechanism to reduce 
service pressures and associated costs 
from inappropriate use of Emergency 
Department services. 1 The majority of OOH 
care in the UK is provided by general prac- 
titioners (GPs), often in conjunction with 
emergency medical and nursing practi- 
tioners, and organised by private providers 
or primary care trusts. Patients typically 
initiate contact through a telephone triage 
system. Those requiring face-to-face consul- 
tations, but not urgent hospital referral, are 
seen by a GP at a local clinic or in the 
patient's home. 
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Patients who contact OOH services are more likely to 
have acute problems than those who are seen in daytime 
primary care. Therefore, GPs working in OOH are likely 
to meet a higher number of acutely ill patients than in 
their routine practice. Mostly these are 'new' patients, 
that is, presenting to GPs who are unlikely to know them 
and with little or no access to their GP or hospital 
records. The combination of lack of continuity and 
high incidence of acute illness provides a potentially 
hazardous practice environment, in which errors can 
occur. Although the incidence of significant clinical 
error is unknown, high profile adverse incidents have 
received significant media attention 3 and prompted 
a fundamental review of OOH service provision in the 
UK. 4 Recommendations focused on clinical governance, 
more effective performance management, better team 
work and making patient records more accessible. In 
addition, they have highlighted staff development issues 
such as the possible use of checklists, regular feedback, 
improving GP training for the specific demands of OOH 
and individualised audits. 

However, the ways in which GPs make clinical deci- 
sions in OOH has received little attention. Yet, how 
clinical decisions are made under these circumstances is 
currently unknown, and little is known of the factors that 
affect safety and referral patterns. We therefore exam- 
ined clinical reasoning (CR) and decision making in an 
OOH setting, with the aim of gaining insights into how 
GPs manage risk while keeping patients safe in this 
environment and focusing on factors they may improve 
the delivery of services. 

METHODOLOGY 

Studies of CR describe the reasoning processes 
employed during a consultation.' We have previously 
published a model of CR based on our work with GPs. 6 
The assumption behind the model was that under- 
standing the reasoning process was essential for fruitful 
reflection on clinical practice. The model highlights key 
areas of knowledge and critical value judgements that 
are used in the clinical encounter. We used this model, 
which is derived from the dual theory of cognition, 7 as 
the framework for a qualitative study of OOH practice. 8 
We are not aware of other studies that have tested the 
applicability of similar theory-based models to real-life 
consultations, though generic theoretical models have 
been proposed. 9 

The dual theory posits two systems interacting with 
each other. The non-analytic system 1 provides the fast 
response, while the reflective system 2 is slow and may 
override the original response. There is rapid recogni- 
tion of a salient feature of the case, 10 which may stem 
from the context provided by what is already known 
about the patient, physical appearance or critical cues 
specific to a presentation. This links to pre-existing 
mental representations, based on theory and experience 
built up of multiple exposures to similar cases. While 
some aspects of the salient feature can be described by 



the clinician, others may appear no more than a 'sense of 
alarm or reassurance', described as 'gut feelings' by 
Stolper et a/, 11 which impact on the reasoning and deci- 
sion-making process of GPs. This combination of recog- 
nition and gut feelings leads to a limited search for more 
cues. System 2 represents reasoning through the initial 
response vis-a-vis the rules provided by the theory of the 
profession and may lead to correcting or overriding 
system 1. In cases where there are no salient features 
perceived, system 2 takes over in place of the absent fast 
response. 7 The initial judgements provide the frame for 
the direction of the search for further information. 
Drivers prioritise the search, generally aimed at risk 
avoidance with the focus on ruling diagnostic options out 
or in, through searching for a few highly critical cues that 
support or negate the original frame. A necessary aspect 
of this kind of reasoning is a stopping rule for data 
collection, but we have not been able to find any research 
that describes them in a clinical setting. 12 

Setting and cohort 

We conducted a 2-month study in Oxfordshire, UK, 
where OOH care is provided by a combination of 
different services. Most weekend, evening and holiday 
care is provided by GPs employed by the primary care 
trust. Overnight services are provided by a private 
provider. Initially, we invited all 62 GPs who regularly 
worked in the OOH service organised by the primary 
care trust. We asked them to participate in 30 min 
interviews about two recently seen OOH cases. We chose 
30 min to minimise the pressure on busy work schedules. 
From past experience, most GPs would be interviewed 
before or after consulting and often during their brief 
lunch breaks. The time allocated was deemed sufficient 
to capture the data on the issues on which we were 
focused. Information about the study was kept to 
a minimum to reduce respondent bias. We obtained 
positive responses from 29 (47%) GPs and interviewed 
21 (34%) GPs, who provided a total of 42 cases. Four GPs 
had been in practice <5 years, nine for 5—20 years and 
eight for > 20 years (range <1— 37 years). In our expe- 
rience and in reports in the literature, busy professionals 
are relatively difficult to persuade to find time for an 
interview. Therefore, one is generally unlikely to find 
random cohorts, the participants coming through word 
of mouth and professional contacts. The only variable we 
would expect to have an impact for this study was length 
of experience, and in this respect, our cohort was rela- 
tively well represented. 

Design and data collection 

To provide face validity to our findings, we used recently 
seen cases, rather than case vignettes in a laboratory 
setting. Participants were asked to describe the presen- 
tation of two new patients who had consulted with them 
during their most recent OOH session and interviewed 
within 48 h of their session. They were asked to discuss 
a straightforward case and a more demanding one. 
After the first few interviews, it became apparent all 
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straightforward cases offered were young children with 
diarrhoea or raised temperature. Subsequent interviews 
excluded this group of presentations. Most GPs referred 
to computer printouts of the cases during the interviews. 
All interviews, conducted by a single researcher (JB), 
were analysed, audiotaped and transcribed. The GPs 
were asked to describe the presentation of the patient 
and told that there would be interruptions to clarify what 
went on in their mind during the consultation. The 
semi-structured interviews used open-ended questions 
and focused on each step of the model (appendix 1). 

The literature suggests that the greatest impact on 
decision making occurs at the first step which frames the 
direction of the process and at the final step of diag- 
nostic closure when immediate data collection stops and 
where most of the errors are likely to occur. 13 14 Our 
focus for the first step was to what extent GPs could 
describe the salient feature that would lead to recogni- 
tion of the case. We specifically did not attempt to dig 
deeper or look for gut feelings, at the same time 
acknowledging their prominence in the diagnostic 
process. On the other hand, we explored these instinc- 
tive aspects at the stage of closure. This relates to our 
particular interest in this step. The scope of the study 
would not allow us a full exploration of all facets of 
each step. 

Data analysis 

The audiotapes were used to develop brief typed 
summaries of each case to gain a sense of the processes; 
these were then used to build a structure for the analysis. 
This was carried out with NVivo software. Two 
researchers searched the text independently for under- 
lying themes in relation to our conceptual framework. 
Categories were coded according to emerging themes 
and added to or changed as new concepts emerged. 15 
Any differences in interpretation were discussed, and 
consensus was reached on clustering to common 
themes. 

We divide the results into five areas: (1) the partici- 
pants' perceptions of the OOH environment and its 
impact on practice, (2) characteristics of a case to make 



it straightforward or demanding, (3) the process of 
reasoning in this context, (4) prerequisites for diag- 
nostic closure and (5) learning points from OOH 
consultations for GP trainees. Within each area, we 
report on common themes raised by the participants. 

RESULTS 

All participants indicated that our questions were easy 
to understand and many remarked that it was a good 
way for them to reflect on their practice. The analysis 
was consistent with previous findings on the dual 
process of reasoning, where instant recognition, 
followed by reasoning and value judgements lead to 
a search for a limited number of cues prior to diagnostic 
closure. (Direct quotes from the interviews appear in 
italics.) 

Key perceptions of OOH environment and its impact on 
practice 

The most powerful themes to emerge related to dealing 
with urgent potentially high-risk cases, trying to keep 
patients out of hospital, the concept of 'fire fighting', 
time pressures and working in an unfamiliar environ- 
ment (table 1). This challenging environment impacted 
on their practice in a number of ways. It was often 
difficult to work out whose agenda they had to deal with, 
the patient's or concerned relatives or carers who had 
contacted the OOH service. Their approach to the 
patient was to deal with immediate problems only, rather 
than the usual holistic approach, as described by one GP: 
basically you 've got to decide whether this chap is safe. Lack of 
feedback and follow-up was the rule: you don't really tend 
to get any feedback at all unless something has gone horren- 
dously wrong (table 2). 

What makes a case straightforward or demanding? 

There were a number of well-defined characteristics 
thath made presentations easy or difficult to deal with. 
The focus was very much on illness severity, and 
a severely ill patient was seen as straightforward, as they 
just required hospital transfer: It's easy when somebody is 
terribly unwell. ..going to admit them to hospital. Oldery 



Table 1 Participants' perceptions of out of hours (OOH) environment (quotes in italics, numeric refers to interviewee ID). 
Emerging themes to question: How is the OOH environment different from normal practice and how does it impact on your 
practice? Frequently these comments came up spontaneously 


Theme 


Example 


Avoiding hospital referral 

Fire fighting 

Time pressure 
Practising in an unfamiliar 
environment 
High-risk patients 


avoid hospital... indication [to refer only if] either further assessment is required or there 
is some definite intervention that the hospital can provide (11); 
intravenous fluids which of course can't be given in the GP context (13); 
only deal with the immediate situation that's sort [of] a bit of fire fighting (12); 
what can we do to make sure he's safe to be seen by his GP in the morning or do 
we need to admit him tonight (3); 

not doing the other patients a service if they are sitting out there waiting (19); 
easier to practise medicine in your own environment (5); 

see a high proportion of high risk patient in out of hours... frightening sometimes (21); 
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Table 2 Impact of out of hours (OOH) environment on practice 



Theme 



Examples 



Lack of background 
information and lack 
of clarity about agenda 
Approach to patient 

Lack of follow-up 
Lack of feedback 

Attitude to work 



Challenging because you don't know the patients (9); whose agenda? (6); you don't have 
access to the records (16); 

I'm not quite so patient centred... a bit of fire fighting ...don't want to miss critical things and 
if there's a red flag symptom that comes up. ..[it] doesn't need to be acted upon (15); 
The sad thing about out of hours is that you don't follow-up on most of your patients (18); 
in reality as an out of hours presentation... it is quite difficult to find out what the discharge 
diagnosis was (17); 

a rewarding part of my work. . . best provided by experienced general practitioners (11); 



patients were often difficult because of the complex 
nature of multiple pre-existing diseases in the absence of 
background information. Children and young adults 
were seen as straightforward with well-defined single 
conditions: Easy cases tend to be younger people with less 
complicated medical histories and where serious things are less 
common. Familiarity with the condition helped, while 
falling between the two extremes tended to lead to lack 
of confidence (table 3). 

The process of CR 

In most cases, there were instantaneous formulations, 
for instance, on seeing a 2-year-old child with 2 days of 
fever, runny nose, coughing and pulling her left ear: the 
likelihood is that it was probably a upper respiratory tract 
infection. . .with otitis media. The salient features that 
brought these to mind were generally recognised: had all 
the signs of a grizzly toddler tugging at the left ear, yep have to 
exclude an otitis media. An inability to make such instan- 
taneous judgement made the case difficult and required 
data clarification, which was not always achieved. Once 
presentation issues were clarified, early formulations 
were associated with brief lists, generally headed by the 



most high-risk clinical conditions. This drove the search 
for one or two critical cues to rule out these. There was 
a particular focus on red flags which potentially indi- 
cated severe illness: she's quite dry... her oxygen saturation is 
a little bit low... I'm really thinking about, is this lady 
compromised because of her diarrhoea? (table 4). 

Prerequisites for diagnostic closure 

Diagnostic closure occurs when the clinician stops 
immediate data collection and makes a management 
decision based on the available information. A rule of 
thumb used by one GP defined the issue: dividing people 
into two groups. . . on the basis of one question: is it serious or 
potentially serious or not? Confidence to stop and make 
a decision was then assessed using a number of criteria as 
in table 5. 

Confidence that appropriate threshold had been 
reached was rarely stated numerically by participants, and 
when asked to assess this way, it was generally over 90%, 
but some were not willing to go beyond a descriptive 
assessment. As a prerequisite to closure, GPs depended on 
a combination of three criteria being met: (1) Confidence 
in ruling out a list of diseases, with clarity about illness 



Table 3 Key themes to differentiate between straightforward and demanding presentations. Question: Can you explain why 
you felt that Case 1 was straightforward and second one more demanding or difficult? (S: straightforward, D: demanding) 



Theme 



Example 



Salient feature rapidly 
recognised or unclear 
Familiar or not 



Age group 



Degree of ongoing 
uncertainty 

Severity of illness 



S: fine within about one minute of seeing the baby just by looking (3); 
D: vague and there was sort of lack of symptoms a lack of I suppose a good history (4); 
S: just the run of the mill you know straight forward diagnosis that you see commonly 
everyday (10); 

D: uncomfortable because I had not seen a rash quite like that before (7); 
S: easy presentations occur in young people with no medical history and whom you don't 
have to worry at all about referring to old notes (15); 
D: chronic ongoing problems where lots of doctors (16); 

S: it's about the degree of risk that I take. . . it is about the degree of confidence that I have 
whether I can help them or not (14); 

D: / had a degree of uncertainty about the second case that I didn't have about the first one (12) 
S: Global impression that child is well [or the other] be dead in a few hours (1); If not serious, 
may or may not know the diagnosis but does not matter (6); you cannot make a definite 
diagnosis, no red flags (13); 

D: more difficult I think the fact that there was a longer differential. . . the stakes were 
higher. . . I had a degree of uncertainty ( 12); 
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Table 4 Process of reasoning in this context. Question: Can you describe what was the main driver of the process? Often 
these statements were made spontaneously during the interview. (Digit after stop refers to 1st or 2nd case for interviewee.) 


Themes 


Example 


Clarify issues and whose 
agenda to deal with first 

Looking for critical cues 
and red flags 

Process of ruling out serious 
illness first 

Process of ruling in common 
and safe condition 

Reassure patient and carers 

Risk assessment 

Used another opinion 


S: think firstly you have to identify what the real issue is (6. 1); 

D: firstly, what real issue is, in OOH may be very problematic (1.2); too vague to 

formulate any thoughts (4.2); 

S: my main concern then was about her hydration (15. 1); 

D: Especially fast resp rate without infection (1.2); wanted to establish that she was 
not confused (2.2); 

S: 1 sort of work backwards rather than forwards and so 1 want to rule out the serious 

things (16.2); 1 want to make sure they are systemically well (15.1); 

D: First thing was likely to be the usual just rule out anything serious (1.1); 

S: rule in, since nil high risk (4. 1); purpose was to confirm the diagnosis which she 

had given me looking for evidence to support... hypothesis (12.1); 

D: initially 1 started with a net looking for the common causes (17.2); getting to the 

correct diagnosis and keeping her safe (20.2); 

S: addressing the patient's concerns (15. 1); 

D: my main concern was safety,... how could 1 keep her as safe as possible while 
trying to respect her autonomy (10.2); 

S: [ketotic pregnant woman]: probably 90% certain that it was a urine infection, 1 would 
say 100% certain in this particular case that admission was needed (13.) ; 1 was just 
slightly concerned that he was starting something more serious so 1 said if his 
temperature went up persistently. . . or if he became breathless. ..he should contact 
back again and that there was somebody on the out of hours all weekend (14. 1); 
D: err on the side of caution. . . threshold to me feels like a fairly nebulous thing. . . the 
clinical state of the patient was surprisingly good (5.2); 1 was reasonably confident. . . 
there was this possibility that we would be able to [treat] her orally (11.2); 
well 1 actually got somebody else in to have a look at it (7.2); speaking to the 
medical registrar at the hospital... the other deciding factor (10.2); 




severity: I'd reached the end of what I could do. (2) Meeting nostic uncertainty is prominent and the patient's condi- 
patient's needs: I'd relieved the patient's anxiety that was the tion may deteriorate unexpectedly. 16 It was consistently 
main issue at stake. (3) An ability to set up a good safety net: stressed during our interviews: mum was,... supportive,... 
a gold star safety net in this case. This is described as safety netting was pretty major, and as a consequence, GPs 
particularly important in emergency settings where diag- went to great lengths to set one up if necessary. 


Table 5 Prerequisites for closure of diagnostic process. Question: Can you try and explain what made you stop the process at 
this particular stage? 


Themes 


Example 


Perception of patient systemically 
well or unwell 

Ruled out urgent or severe illness 

Ruled in non-urgent diagnosis 
with satisfactory confirmation 

Red flags present or absent 
Set up good safety net 

Responded to patient's needs 


Well: / didn't think she was medically kind of unwell (4); vital signs were good (13); 
Unwell: She looked ill and she was vomiting profusely (19); 
urgent problems... excluded... don't really know what's going on now, but 1 didn't 
think it was anything serious (3); 1 felt I'd done more excluding than 1 had done 
confirming (17); 

know it's to be one of the simpler things but you have to ask the questions, and 
1 guess how much is enough 1 mean at the point when I'm satisfied that there is 
significantly more confirmation of one of the simpler things than one of the 
nasties (5); the history and the examination coalesce on a simple diagnosis (11); 
ketotic... don't think 1 could have done anything else (13); her gait was 
staggering (21); 

safety netting because everyone makes mistakes, (1 9); 1 normally very 
scrupulous about safety netting because I'm paranoid about missing things (7); 
a gold star safety net in this case. . . driver knew the patient (2); 
...in terms of the diagnosis 1 think 1 was less than 50% sure..., but in terms of 
what was best to do for the patient... towards 100% (18); 1 hadn't succeeded in 
allaying his anxieties (6); Stop since good explanation that is rule in and also 
consistent with parent expectations (7); 
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Learning points for GP trainees 

This was the last question we asked about each case. 
It turned out to be particularly useful as it gave GPs 
time to pause and reflect on what they regarded as 
particularly significant training issues relating to 
that consultation. Emerging themes divided into (1) 
training needs on how to deal with patients, stressing 
the need to develop rapport despite the intense time 
pressures; (2) learning to deal with uncertainty: 
think about how certain am I, how certain do I need to be to 
send him home and (3) learning about common presen- 
tations with a focus on critical cues to exclude severe 
illness and a few high-risk diseases: what are the danger 
signs you would look out for; specific symptom questions you 
want to ask (table 6). 

DISCUSSION 

To our knowledge, this is the first study to show how the 
context of OOH practice impacts on the CR process in 
primary care. Participants were aware that their behav- 
iours and decisions were affected by the environment. In 
line with the perceived purpose of OOH care, objectives 
were to keep patients safe, respond to their needs and 
reduce unnecessary hospital attendances. Their expec- 
tation was to deal with urgent and potentially high-risk 
issues only, avoiding hospital admissions unless patients 
were at unacceptable risk. 

GPs found practice in OOH rewarding but identified 
a number of issues that made this a high-risk clinical 
environment. The most prominent of these include 
a lack of background information in the form of medical 
records and complex multisystem chronic disease 
presentations where it was difficult to define the perti- 
nent issues. Success depended on clinical skills to 
develop rapport with patients and recognise their 
concerns. Lack of regular feedback on clinical perfor- 
mance was raised repeatedly. 

Our findings were consistent with the dual theory of 
cognition. In line with system 1, there was instant 
recognition on finding salient features of the presen- 
tation. Even before GPs knew the presenting complaint, 
they were constantly on the lookout for severe illness 



(tables 1 and 3), using red flags (table 4) and well-tried 
markers of systemic illness (table 5). Lists and markers 
were derived from personal experience, and the novice 
would struggle in their absence (table 6). These early 
cues would frame the immediate search for further 
cues. The major driver was to exclude serious disease 
and severe illness (table 4). Ruling out is commonly 
used in everyday practice, but it was practically 
uniform in OOH. It was surprising how often this was 
seen as 'working backwards rather than forwards'. 
System 2 that represents correcting or overriding system 
1 was used in many but not all cases. For instance, 
ruling in first was used only occasionally, in straight- 
forward familiar presentations, and then followed by 
ruling out in most cases. Once serious disease, severe 
illness or the risk of abrupt deterioration were 
excluded, the precise diagnosis became less important 
and could be left for another day. Such decisions, as 
they often required reflection, represented system 2. 
Diagnostic closure that occurs after a clinician has 
weighed up all the options and needs to choose 
a specific course of action, in all but the most clear-cut 
presentation would involve such reflection (table 5). 
This is the point where clinical errors crystallise, and 
therefore, understanding the rules and thresholds for 
stopping provides insights into the causes of diagnostic 
error. Little is known about how diagnostic closure 
occurs in most clinical specialties and settings, 
including primary care. The OOH setting provided 
a unique opportunity to gain insights into diagnostic 
closure, as clinical care is associated with clear-cut end 
points, not often seen in other practice settings. The 
stopping rules used by GPs involved three criteria: first, 
have high-risk conditions and severe illness that require 
urgent hospital attention been excluded? Second, have 
I responded directly to the patient's needs? Third, is 
there a reliable safety net in place, given the high-risk 
environment where error may be more likely than in 
a familiar situation? 

The basis of individualised judgements about accept- 
able confidence levels for the stopping rules were diffi- 
cult to define and as one GP put it 'feels like a nebulous 



Table 6 Trainee take home messages. Answers in response to question: What sort of message would you like your trainee to 
take home from this consultation? 



Theme 



Example 



Patient-related issues 
Case-specific learning 



The process 



Dealing with uncertainty 



Take time to establish a rapport (2); listen to the patient's concerns ( 15); 
this is how chest infections present in the elderly (3); common things that can cause sudden 
deterioration in elderly patients (13); you know these kinds of vague symptoms in an elderly 
patient probably you would want to rule out anything sinister or acute (4); I'd want them to 
sort of look up investigations for unexplained bone pain and what would be the key questions 
for red flags and what blood tests they might want (9); 

looking at the global perspective of the child (1); I said to him look I really don't find signs of 
anything serious or anything potentially serious just now (6); the take home message is the 
safety net (5); 

uncertainty... sometimes you don't know... talk them through the probability side of things 
I would discuss the things that lead me towards one way or another (7); 



6 



Balla J, Heneghan C, Thompson M, et al. BMJ Open 2012;2:e000414. doi:10.1136/bmjopen-2011-000414 



Clinical decision making in a high-risk primary care environment 



thing . This is consistent with a previous study, which 
used an end point of variation in hospital referral rates 
and concluded that the individual nature of learning 
from experience leads to a variation in approach to risk 
tolerance and associated confidence levels for decision 
making. 18 We believe that our findings are consistent 
with the work of Gabbay and le May, 19 who suggested 
that multiple exposures to clinical experience, 
embedded in a social context lead to a set of 'internal- 
ised, collectively reinforced and often tacit guide- 
lines... malleable to deal both with individual patients' 
needs and the multifarious factors that come into reck- 
oning when making decisions.' 

Strengths and limitations of the study 

This is the first study that we are aware of that has 
explicitly explored the diagnostic reasoning that occurs 
in OOH primary care and is likely to be relevant to other 
countries with similar systems of primary care. The 
design of the study is based on a strong theoretical 
framework provided by the dual theory of cognition. 
This has been shown to have generalisability to human 
reasoning irrespective of the domain. 7 There are also 
good theoretical models suggesting that clinical thinking 
can be modelled within this framework. 9 The study also 
has face validity using recently seen cases, where partic- 
ipants were not made aware of the content and purpose 
of the interviews. 

Limitations relate to a number of issues. (1) Sampling: 
participants were not chosen at random but consisted of 
self-selected individuals who made up a third of a cohort 
we approached. This is an almost universal problem with 
studies that involve busy and senior clinicians. We asked 
participants to select the two most recently seen patients 
who met the selection criteria. We cannot be certain if 
these rules were always applied. We have not performed 
similar patient-based studies in other settings in the UK 
or in countries with different health systems, and to this 
stage, the research has not included other specialty 
groups. (2) There are concerns regarding the validity of 
think-aloud protocols, but they have been considered 
appropriate for studies of cognition in natural settings as 
used here. 20 (3) We designed the study to allow us to 
focus on specified steps of the CR process, for example, 
closure. This means that some steps could not have full 
coverage of issues, such as some of the crucial intuitive 
aspects of the early stages of diagnosis. This will require 
further qualitative work. (4) Another issue relates to 
triangulation. These qualitative studies lead to the 
development of inventories suitable for triangulation 
through quantitative studies. 

We believe that the dual theory of cognition is the 
closest model to how people actually think in real-life 
scenarios and this makes it an appropriate way to study 
reasoning in a clinical setting. 13 On account of the 
strong theoretical base, our findings, albeit with 
different emphases, are likely to have some degree of 
generalisability to other specialties and health systems. 



Implications for practice 

High-quality OOH care is important to the functioning 
of the health service and particularly the interface 
between primary care, emergency care and acute 
hospital care. While the results of our investigation 
provide a unique insight into the clinical decision 
making that occurs in OOH, we have several practical 
recommendations based on our research which could 
potentially be implemented with minimal additional 
resources. 

Recognising the limitations of our study, we also find 
that our conclusions are consistent with the literature on 
the importance of feedback on performance and 
settings for fruitful reflection on experience. We believe 
that given this congruence between the relevant litera- 
ture and our proposals, our recommendations merit 
cautious acceptance and evaluation. 

First, at the organisational level, an automated, regular 
and timely system of feedback to GPs and other clinical 
staff practising in the OOH setting is essential. Providing 
clinical care in an environment almost devoid of clinical 
feedback (apart from when there are complaints or 
serious events) is contrary to the type of practice that GPs 
are trained to work in. One of the mainstays of primary 
care practice is the longitudinal relationship with 
patients, so that information gathering and clinical 
decision making occur over several encounters. In the 
OOH setting, this process becomes compressed and is 
compounded by high patient flow. Since the vast majority 
of OOH care is provided by local GPs and other staff, an 
initial step would be to provide routine feedback on the 
outcomes of their clinical decisions. Since most OOH 
services are now computerised, this could be imple- 
mented with little additional resource. These systems 
need to be implemented with sensitivity, recognising the 
risk of creating anxiety and dislike of being monitored. 
Feedback needs to be seen as nonjudgemental and to be 
used by the recipient and trusted colleagues only. The 
importance of regular feedback on performance to gain 
expertise is well accepted in the literature. 21 

Second, for the individual, in line with the principles 
of fruitful learning from experience, we suggest that 
peer and expert clinician support with regular meetings 
and feedback on recent cases should be implemented. 
The educational literature provides good evidence that 
practice with feedback enhances deep learning, which 
goes with improved understanding of subject matter. 22 
GPs would then be in a position to reflect, calibrate their 
decision making and update their mental constructs, all 
necessary for good practice. Some GPs noted that they 
previously had used informal coffee breaks to do this, 
but the increasing numbers of patients attending OOH 
now precluded this. However, such informal systems are 
probably inadequate, and we propose that OOH practice 
should experiment with formal opportunities for 
reflection in a safe and supportive environment. 

Third, our research has implications for training GPs, 
whose requirement for OOH training and experience is 
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fairly modest. Individual mentoring and group discus- 
sions need to focus on (1) how to develop rapport 
with new and anxious patients; (2) how to deal 
with uncertainty and (3) common presentations with 
a focus on critical cues to exclude severe illness and the 
relatively few high-risk diseases. We believe that the 
model of deliberate practice, as described by Ericsson, 23 
is likely to be a suitable model for training. The clinician 
seeks out cases in areas of perceived weakness, and 
practice is associated with immediate individual feed- 
back and group discussions. To this stage, excellent 
improvements have been demonstrated in interven- 
tional specialties but not in primary care. However, in 
view of the confluence of the principles of deliberate 
practice and deep learning approaches, we recommend 
the evaluation of deliberate practice training models for 
trainees. 

We also draw attention to the ongoing difficulties in 
accessing GP or hospital records from OOH settings; 
a situation that we acknowledge is unlikely to change 
without the continued significant investment in the NHS 
IT infrastructure. 

Implications for future research 

We recognise that our study is a starting point for further 
research in this area. To explore the generalisability of 
our findings to other OOH or alternative emergency 
settings, our studies will need to be replicated with 
appropriate cohorts of GPs and other clinicians working 
in such environments. We also propose to develop 
a quantitative questionnaire based on our qualitative 
studies and on leads that we find in the literature. Such 
studies, extending beyond our cohort, may then provide 
data to allow greater generalisability of the results. We do 
not know if the practice changes that we propose will 
lead to better clinical care and this needs separate eval- 
uation. Finally, investigation of the rules clinicians use 
for diagnostic closure may give us insights into the root 
causes of clinical error. We are at this stage in the process 
of developing a qualitative study, examining diagnostic 
error in general practice. 

Conclusions 

This study provides the first evidence of how working in 
an OOH environment impacts on CR and decision 
making. GPs identify OOH as a potentially high-risk 
clinical environment and adapt their practice approach 
to this setting. Lack of access to GP or hospital records is 
a major concern. Our study provides further support to 
the literature on reflective practice and educational 
programmes in clinical settings. We suggest that 
improving feedback to GPs about their clinical decisions 
and providing opportunities for reflection on OOH 
practice may be valuable for ongoing review and 
improvement of clinical practice. The findings provide 
guidance to the development of GP training 
programmes to cover the specific needs of OOH. 
Exploring further the rules and criteria that clinicians 



use for diagnostic closure may give us insights into the 
root causes of clinical error. 

Contributors JB designed the study, conducted the interviews, analysed the 
data and produced the first draft to final document. CH and MT conceived the 
project and were involved with design and all stages of writing the drafts and 
final document. MB participated in the interviews and analysis as well as 
reviewing the drafts and final document. 

Funding This research received no specific grant from any funding agency in 
the public, commercial or not-for-profit sectors. 

Competing interests None. 

Ethics approval This is a quality improvement study involving volunteer 
general practitioners who cannot be identified. 

Provenance and peer review Not commissioned; externally peer reviewed. 
Data sharing statement The original transcripts are available. 



REFERENCES 

1 . Zwart DL, Van Rensen EL, Kalkman CJ, etal. Central or local incident 
reporting? A comparative study in Dutch GP out-of-hours services. Br 
J Gen Pract 201 1 ;61 : 183-7. 

2. Kings Fund. Improving the Quality of Care in General Practice. 
Independent Inquiry Into the Quality of Care in General Practice in 
England, http://www.kingsfund.org.uk/publications/gp_inquiry_report. 
html (accessed Jul 2011). 

3. Cosford PA, Thomas JM. Safer out of hours primary care. BMJ 
2010;340:c3194. 

4. Colin-Thome D, Field S. General Practice Out-of-Hours Services: 
Project to Consider and Assess Current Arrangements. http://www. 
dh.gov.uk/en/Publicationsandstatistics/Publications/ 
PublicationsPolicyAndGuidance/DH_111892 (accessed Jul 2011). 

5. Norman G. Research in clinical reasoning: past history and current 
trends. Med Educ 2005;39:41 8-27. 

6. Balla Jl, Heneghan C, Glasziou P, et al. A model for reflection for 
good clinical practice. J Eval Clin Pract 2009;15:964-9. 

7. Kahneman D. Maps of Bounded Rationality: A Perspective on 
Intuitive Judgment and Choice, http://www.nobelprize.org/ 
nobel_prizes/economics/laureates/2002/ (accessed Jul 2011). 

8. Pope C, Ziebland S, Mays N. Qualitative research in health care. 
Analysing qualitative data. BMJ 2000;320:1 14-16. 

9. Croskerry P. A universal model of diagnostic reasoning. Acad Med 
2009;84:1022-8. 

10. Kahneman D, Klein G. Conditions for intuitive expertise: a failure to 
disagree. Am Psychol 2009;64:515-26. 

1 1 . Stolper E, Van de Wiel M, Van Royen P, et al. Gut feeling as a third 
track in general practitioner's diagnostic reasoning. J Gen Intern Med 
2011;26:197-203. 

12. Gigerenzer G, Goldstein DG. Betting on one good reason: take the 
best heuristic. In: Gigerenzer G, Todd PM, eds; ABC Research 
Group. Simple Heuristics that Make Us Smart. New York: Oxford 
University Press, 1999. 

13. Norman G. Dual processing and diagnostic errors. Adv Health Sci 
Educ 2009;14:37-49. 

1 4. Graber ML. Educational strategies to reduce diagnostic error: can you 
teach this stuff? Adv Health Sci Educ Theory Pract 2009;14:63-9. 

15. Ritchie J, Spencer L. Qualitative data analysis for applied policy 
research. In: Bryman A, Burgess RG, eds. Analysing Qualitative 
Data. London: Routledge, 2011:173-94. 

16. Neighbour R. The Inner Consultation. Lancaster: MTP Press, 1987. 

17. Murtagh J. Murtagh's General Practice Companion Handbook. 4th 
edn. North Ryde, NSW: McBiggs J, Tang Graw-Hill, 2007. 

18. Calnan M, Payne S, Kemple T, ef al. A qualitative study exploring 
variations in GPs' out-of-hours referrals to hospital. Br J Gen Pract 
2007;57:706-13. 

19. Gabbay J, Le May A. Practice-Based Evidence for Healthcare. 
Abingdon: Routledge, 201 1 . 

20. Frith CD, Lau HC. The problem of introspection. Conscious Cogn 
2006;15:761-4. C. Teaching Quality Learning at University. 
Maidenhead: Open University Press, 2007. 

21. Schiff GD. Minimizing diagnostic error: the importance of follow-up 
and feedback. Am J Med 2008; 1 2 1 (5 Suppl):S38-42. 

22. Biggs J, Tang C. Teaching Quality Learning at University. 
Maidenhead: Open University Press, 2007. 

23. Ericsson KA. Deliberate practice and acquisition of expert 
performance in medicine and related domains. Acad Med 
2004;79(10 Suppl):S70-81 . 



8 



Balla J, Heneghan C, Thompson M, etal. BMJ Open 2012;2:e000414. doi:1 0.1 136/bmjopen-201 1-000414 



